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LEARNING CURVE AND RATE ADJUSTMENT MODELS: 



COMPARATIVE PREDICTION ACCURACY UNDER VARYING CONDITIONS 

ABSTRACT 

Learning curve models have gained widespread acceptance as a 
technique for analyzing and forecasting the cost of items produced 
from a repetitive process. Considerable research has investigated 
augmenting the traditional learning curve model with the addition 
of a production rate variable, creating a rate adjustment model. 
This study compares the predictive accuracy of the learning curve 
and rate adjustment models. A simulation methodology is used to 
vary conditions along seven dimensions. Forecast errors are 
analyzed and compared under the various simulated conditions using 
ANOVA. Overall results indicate that neither model dominates; each 
is more accurate under some conditions. Conditions under which 
each model tends to result in lower forecast errors are identified 



and discussed. 



LEARNING CURVE AND RATE ADJUSTMENT MODELS: 



COMPARATIVE PREDICTION ACCURACY UNDER VARYING CONDITIONS 
INTRODUCTION 

Learning curves have gained widespread acceptance as a tool for 
planning, analyzing, explaining, and predicting the behavior of the 
unit cost of items produced from a repetitive production process. 
(See Yelle, 1979, for a review.) Cost estimation techniques for 
planning the cost of acquiring weapon systems by the Department of 
Defense, for example, typically consider the role of learning in 

the estimation process. The premise of learning curve analysis is 

\ 

that cumulative quantity is the primary driver of unit cost. Unit 
cost is expected to decline as cumulative quantity increases. 

There is general acknowledgement that cumulative quantity is 
not the only factor that influences anit cost and that the simple 
learning curve is not a fully adequate description of cost 
behavior. Hence prior research has attempted to augment learning 
curve models by including additional variables (e.g., Moses, 1990) . 
Most attention has been focused on the addition of a production 
rate term. 1 The resulting augmented model is usually referred to 
as a rate adjustment model. 

Conceptually, production rate should be expected to affect unit 
cost because of the impact of economies of scale. Higher 

’One review of the literature pertaining to learning curves 
(Cheney, 1977) found that 36% of the articles reviewed attempted to 
augment the learning curve model in some manner by the inclusion of 
production related variables. 
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production rates may lead to several related effects: greater 
specialization of labor, quantity discounts and efficiencies 
associated with raw materials purchases, and greater use of 
facilities permitting fixed overhead costs to be spread over a 
larger output quantity. Together, these effects work to increase 
efficiency and reduce production cost (Bemis, 1981; Boger and Liao, 
1990; Large, et. al., 1974; Linder and Wilbourn, 1973). However, 
higher production rate does not guarantee lower cost. When 
production rate exceeds capacity, such factors as over-time pay, 
lack of skilled labor, or the need to bring more facilities online 
may lead to inefficiencies and increased unit cost. In short, 
production rate may be associated with both economies and 
diseconomies of scale. 

PRIOR RESEARCH 

Numerous studies, using data on actual production cost 
elements, have been conducted to empirically examine the impact of 
production rate on unit cost. The broad objective of the research 
has been to document rate/cost relationships and determine if 
consideration of production rate leads to improvements in cost 
explanation or prediction. Results have been inconsistent and 
general findings inconclusive. Various studies (e.g., Alchian, 
1963; Cochran, 1960; Hirsh, 1952; Large, Campbell and Cates, 1976) 
found little or no significance for rate variables. Other studies 
did document significant rate/cost relationships (e.g., Bemis, 
1981; Cox and Gansler, 1981). Some research found significant 
results only for particular individual cost elements, such as labor 
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(Smith, 1976), tooling (Levenson, et. al., 1971) or overhead 
(Large, Hoffroayer, and Kontrovich, 1974) . But rate/cost 
relationships for these same cost elements were not consistently 
evident in other studies. When significant, estimates of the 
rate/cost slope varied greatly and the direction of the 
relationship was sometimes negative and sometimes positive (e.g., 
Moses, 1990) . In reviewing the existing research on production 
rate. Smith (1980) concluded that a rate/cost relationship may 
exist but that the existence, strength and nature of the 
relationship varies with the item produced and the cost element 
examined . 

Several explanations for these varying, inconclusive empirical 
results can be offered: 

(a) Varying results are to be expected because rate changes can 
lead to both economies and diseconomies of scale. 

(b) Production rate effects are difficult to isolate 
empirically because of colinearity with cumulative quantity 
(Gulledge and Womer, 1986) . 

(c) Researchers have usually used inappropriate measures of 
production rate leading to misspecified models (Boger and Liao, 
1990) . 

(d) The impact of a production rate change is dominated by 
other uncertainties (Large, Hoffmayer, and Kontrovich, 1974), 
particularly by cumulative quantity (Asher, 1956) . Alchian (1963) , 
for example, was unable to find results for rate adjustment models 
that improved on the traditional learning curve without a rate 
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parameter. 



OBJECTIVE OF THE STUDY 

The prior research suggests that consideration of production 
rate sometimes improves cost explanation, but not always. The 
prior research suggests that a traditional learning curve model 
sometimes is preferable to a rate adjustment model, but not always. 
The prior research provides little guidance concerning the 
circumstances under which explicit incorporation of production rate 
into a learning curve model is likely to lead to improved 
explanation or prediction. This issue is important in a number of 
cost analysis and cost estimation situations. Dorsett (1990), for 
example, describes the current situation facing military cost 
estimators who, with the military facing budget reductions and 
program stretchouts, are required to rapidly develop weapon system 
acquisition cost estimates under many different quantity profiles. 
One choice the cost analyst faces is between using a rate 
adjustment model or a traditional learning model to develop estimates. 2 

The objective of this paper is to address the following broad 
issue: Under what circumstances is it beneficial to explicitly 



2 Two other techniques for making cost estimates when 
production rate changes are also mentioned by Dorsett: curve 
rotation, which involves an ad hoc upward or downward revision to 
the slope of the learning curve, and the use of repricing models 
(e.g., Balut, 1981; Balut, Gulledge, and Womer, 1989) which adjust 
learning curve estimates to reflect a greater or lesser application 
of overhead cost. Dorsett criticized curve rotation for being 
subjective and leading to a compounding of error when the 
prediction horizon is not short. He criticized repricing models 
because they must be plant-specific to be effective. 
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consider production rate in cost analysis? That broad issue is 
addressed by focusing more narrowly on the following research 
question: Under what circumstances do rate adjustment models 

outperform traditional learning curve models? Performance here 
means the ability to provide accurate predictions of future cost. 3 
RESEARCH APPROACH 

Operationally the research question implies a comparison of the 
predictive accuracy of two competing cost estimation models. The 
two competing models were as follows: 

The traditional learning curve model, which predicts unit cost 
as a function of cumulative quantity 4 : 

C L = aQ b (1) 

where 

C L = Unit cost of item at quantity Q (i.e., with learning 
considered) . 

Q = Cumulative quantity produced. 

a = Theoretical first unit cost. 

b = Learning curve exponent (which can be converted to a 
learning slope by slope = 2 b ) . 



3 For many of the studies cited in the section on prior 
research, performance was assessed in terms of a model's ability to 
ex post, statistically account for the variance in an actual cost 
series. The problem of "overfitting" a model to data always exists 
when regression techniques are used to ex post explain a cost 
series. In fact, if higher R 2 is achieved by overfitting, 
predictive ability can be reduced (Wetherill, 1986). 

4 Note that this is an incremental unit cost model rather than 
a cumulative average cost model. Liao (1988) discusses the 
differences between the two approaches and discusses why the 
incremental model has become dominant in practice. One reason is 
that the cumulative model weights early observations more heavily 
and, in effect, "smooths" away period-to-period changes in average 
cost . 
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The most widely used rate adjustment model, which modifies the 
traditional learning curve model with the addition of a production 
rate term: 

C R = aQ^ 0 (2) 

where 

C R = Unit cost of item at quantity Q and production rate per 
period R (i. e., with production rate as well as 

learning considered) . 

Q = Cumulative quantity produced. 

R = Production rate per period measure. 

a = Theoretical first unit cost. 

b = Learning curve exponent. 

c = Production rate exponent (which can be converted to a 

production rate slope by slope = 2 C ) . 

A simulation approach was used to address the research 
question. In brief, cost series were generated under varying 
simulated conditions. The learning curve model and the rate 
adjustment model were separately fit to the cost series to estimate 
model parameters. The estimated models were then used to 
separately predict future cost. The relative accuracy of the two 
alternative future cost predictions was determined. Finally, an 
analysis (ANOVA) was conducted relating relative prediction 
accuracy (dependent variable) to the simulated conditions 
(independent variables). 

There are three main benefits gained from the simulation 
approach. First, factors hypothesized to influence prediction 
accuracy can be varied over a wider range of conditions than would 
be encountered in any one (or many) sample (s) of actual cost data. 
Second, explicit control is achieved over the manipulation of 
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factors. Third, noise caused by factors not explicitly 
investigated is removed. Hence simulation provides the most 
efficient way of investigating data containing a wide variety of 
combinations of the factor levels while controlling for the effects 
of other factors not explicitly identified. 

RESEARCH CHOICES 

There were five choices that had to be made in conducting the 
simulation experiment: 

(1) The form of the rate adjustment (RA) model whose 
performance was to be compared to the learning curve (LC) model. 

(2) The functional form of the cost model used to generate the 
simulated cost data. 

(3) The conditions to be varied across simulation treatments. 

(4) The cost objective (what cost was to be predicted). 

(5) The measure of prediction performance. 

Items (1), (2), (4) and (5) deal with methodological issues. Item 

(3) deals with the various conditions simulated. The discussion of 
item (3) encompasses the major variables examined in the study and 
hence the major issues addressed. Each item will be discussed in 
turn. 

1. The Rate Adjustment Model . Various models, both 

theoretical and empirical, have been suggested for incorporating 
production rate into the learning curve (Balut 1981; Balut, 
Gulledge, and Womer, 1989; Linder and Wilbourn, 1973; Smith, 1980, 
1981; Washburn, 1972; Womer, 1979). The models vary with respect 
to tradeoffs made between theoretical completeness and empirical 
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tractability . Equation 2, described above, was the specific rate 
adjustment model analyzed in this study, for several reasons: 
First, it is the most widely used rate adjustment model in the 
published literature. Second, it is commonly used today in the 
practice of cost analysis (e.g., Dorsett, 1990). Third, in 
addition to cost and quantity data (needed to estimate any LC 
model) , equation 2 requires only production rate data. Thus 
equation 2 is particularly appropriate for examining the 
incremental benefit of attention to production rate. (Other RA 
models offered in the literature require knowledge of still 
additional variables) . 5 In short, equation 2 is the most widely 
applicable and most generally used rate adjustment model. 

2. The Cost Generating Function : A "true" cost function 
for an actual item depends on the item, the firm, the time period 
and all the varying circumstances surrounding actual production. 
It is likely that most manufacturers do not "know" the true cost 
function underlying goods they manufacture. Thus the choice of a 
cost function to generate simulated cost data is necessarily ad 
hoc. The objective here was to choose a "generic" cost function 
which had face validity, which included components (parameters and 
variables) that were generalizable to all production situations, 
and which resulted in a unit cost that depended on both learning 

5 The equation 2 model is particularly applicable in situations 
where a cost analyst or estimator does not have ready access to or 
sufficient knowledge about the cost structure and cost drivers of 
a manufacturer. Examples include the Department of Defense 
procuring items from government contractors in the private sector, 
or prime contractors placing orders with subcontractors. 
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and production rate factors. The following explanation of the cost 

function used reflects these concerns. 

At the most basic level the cost of any unit is just the sum of 

the variable cost directly incurred in creating the unit and the 

share of fixed costs assigned to the unit, where the amount of 

fixed costs assigned depend on the number of units produced. 

UC = VC + FC (3) 

PQ 

where 

UC = Unit cost. 

VC = Variable cost per unit. 

FC = Total fixed costs per period. 

PQ = Production quantity per period. 

The original concept of "learning" (Wright, 1936) involved the 

reduction in variable cost per unit expected with increases in 

cumulative quantity produced. (By definition, fixed costs are 

assumed to be unaffected by volume or quantity.) To incorporate 

the effect of learning, variable cost can be expressed as: 

VC Q = VC 1 (Q d ) (4) 

where 

Q = Cumulative quantity. 

VC Q = Variable cost of the Qth unit. 

VC 1 = Variable cost of the first unit, 
d = Parameter, the learning index. 

Substituting into equation 3: 

UC 0 = VC 1 (Q d ) + FC (5) 

PQ 

Additionally, assume the existence of a "standard" ("benchmark," 
"normal," "planned") production quantity per period (PQ S ) . 
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Standard fixed cost per unit (SFC) at the standard production 
quantity would be: 



SFC = FC (6) 

PQ S 

The production rate (PR) for any period can then be expressed 

as a ratio of the production quantity to the standard quantity: 

PR = P£> (7) 

PQ S 

The second term of equation (6) can then be rewritten as: 

FC = SFC (8) 

PQ PR 

and equation 5 rewritten as: 

UC Q = VC 1 (Q d ) + SFC (PR' 1 ) (9) 



Ir. this formulation it can be seen that total cost per unit is 
the sum of variable cost per unit (adjusted for learning) plus 
standard fixed cost per unit (adjusted for production rate) . This 
model incorporates the two factors presumed to impact unit costs 
that have been most extensively investigated: cumulative quantity 
(Q) and production rate per period (PR) . 6 It is consistent with 
both the theoretical and empirical literature which sees the 
primary impact of learning to be on variable costs and the primary 



6 Smith (1980, 1981), for example, used a model similar to 
equation 9 to explore the effect of different production rates on 
unit cost. Balut (1981) and Balut, Gulledge and Womer (1989) 
construct models based on learning and production quantity to 
assist in "redistributing" overhead and "repricing" unit costs when 
changes in production rate occur. The Balut and Balut, Gulledge 
and Womer models differ in that they determine a learning rate for 
total (not variable) unit cost and then apply an adjustment factor 
to allow for the impact of varying production quantity on the 
amount of fixed cost included in total cost. 
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impact of production rate to be on the spreading of fixed costs 
(Smith, 1980) . Simulated cost data in this study was generated 
using equation 9, while varying values for the variables and 
parameters on the right hand side of the equation to reflect 
differing conditions. 

3. The Simulated Conditions : The general research hypothesis 
is that the relative predictive accuracy of the LC and RA models 
will depend on the circumstances in which they are used. What 
conditions might be hypothesized to affect prediction accuracy? 
Seven different factors (independent variables) were varied during 
the simulation. In the following paragraphs, each factor is 
discussed. A label for each is provided, along with a discussion 
of why the factor may be relevant to prediction accuracy and how 
the factor was operationalized in the simulation. Table 1 
summarizes the seven factors. 

i) Data History (DATAHIST) : The number of data points 
available to estimate parameters for a model should affect the 
accuracy of a model. More data available during the "estimation 
period" should be associated with greater accuracy for both the LC 
and the RA model. 7 But increased data may not impact the accuracy 
of the two models in the same manner. The RA model requires the 
estimation of an additional parameter, and consequently is likely 
to require more data points to avoid unreliable parameter estimates 
and resultant prediction inaccuracy. Hence it may be hypothesized 

7 There are, of course, cost/benefit tradeoffs. The marginal 
benefits of increased prediction accuracy for any model must be 
weighed against the marginal costs of additional data collection. 
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that the RA model will be more sensitive (less accurate) when data 
is lean. 

In the simulation, data history was varied from four to seven 
to ten data points available for estimating model parameters. This 
simulates having knowledge of costs and quantities for four, seven 
or ten production lots. Four is the minimum number of observations 
needed to estimate the parameters of the RA model by regression. 
The simulation focuses on lean data availability both because the 
effects of marginal changes in data availability should be most 
pronounced when few observations are available and because many 
real world applications (e.g., cost analysis of Department of 
Defense weapon system procurement) occur under lean data 
conditions. 

ii) Variable Cost Learning Rate (LEARNRAT) : In the cost 
generating function, learning affects total unit cost by affecting 
variable cost per unit. Past research (Smunt, 1986) has shown that 
the improvement in prediction accuracy from including a learning 
parameter in a model (when compared to its absence) depends on the 
degree of learning that exists in the underlying phenomena being 
modeled; the greater the degree of learning, the more benefit 
gained from inclusion of a learning rate term. This suggests that 
prediction accuracy may depend on the learning rate and hence that 
relative prediction accuracy of the LC and RA models may also 
depend on the learning rate. In the simulation, variable cost 
learning rate (reflected in parameter d in equation 9) was varied 
from 75% to 85% to 95%. Generally, complex products or labor 
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TABLE 1 



INDEPENDANT VARIABLES 



Concept 


Label 




Levels 


Data History 


DATAHIST 1 


4 


7 


10 


Variable Cost Learning 
Rate 


LEARNRAT 


75% 


85% 


95% 


Fixed Cost Burden 


BURDEN 2 


15% 


33% 


50% 


Production Rate Trend 


PROTREND 3 


Level 




Growth 


Production Rate 
Instability/Variance 


RATEVAR 4 


.05 


.15 


.25 


Cost Noise/Variance 


COSTVAR 5 


. 05 


. 15 


.25 


Future Production Level 


FUTUPROD 6 


Low 


Same 


High 



’Number of data points available during the model estimation 
period; simulates the number of past production lots. 

Standard per unit fixed cost as a percentage of cumulative 
average per unit total cost, during the model estimation period. 

3 A level trend means production at 100% of standard production 
for each lot during the estimation period. A growth trend means 
production rate gradually increasing to 100% of standard production 
during the estimation period. The specific growth pattern depends 
on the number of production lots in the estimation period, with 
sequences as follows (expressed as a % of standard) : For DATAHIST 
= 4: 33%, 67%, 100%, 100%. For DATAHIST = 7: 20%, 40%, 60%, 80%, 
100%, 100%, 100%. For DATAHIST = 10: 10%, 20%, 35%, 50%, 70%, 90%, 
100%, 100%, 100%, 100%. 

Coefficient of variation of production rate. (Degree of 
instability of production rate around the general production rate 
trend. ) 

Coefficient of variation of total per unit cost. 

6 "Same" means production rate at 100% of standard for each 
lot produced within the prediction zone. "Low" means production 
rate at 50%. "High" means production rate at 150%. 



intensive processes tend to experience high rates of learning (70- 
80%) while simple products or machine-paced processes experience 
lower (90-100%) rates (Smunt, 1986) . 8 

iii) Fixed Cost Burden (BURDEN) : In theory (and in the cost 
function, equation 9) a change in the number of units produced 
during a period affects unit cost in two ways: First, increasing 
volume increases cumulative quantity and decreases variable cost 
per unit, due to learning. Second, increasing volume increase the 
production rate for a period and reduces fixed cost per unit, due 
to the spreading of total fixed cost over a larger output. Both 
these effects operate in the same direction; i. e., increasing 
volume leads to lower per unit cost. This has led some cost 
analysts to conclude that in practice, it is sufficient to use an 
LC model, letting the cumulative quantity variable reflect the dual 
impacts of increased volume. Adding a production rate term to an 
LC model is seen as empirically unnecessary. 

In principle, if fixed cost was zero, cumulative quantity would 
be sufficient to explain total unit cost and production rate would 
be irrelevant. But as fixed cost increases as a proportion of 
total cost, the impact of production rate should become important. 
This suggests that the relative predictive accuracy of the LC and 
RA models may depend on the amount of fixed cost burden assigned to 
total cost. 

Fixed cost burden was simulated by varying the percentage of 

8 See Conway and Schultz (1959) for further elaboration of 
factors impacting learning rates. 
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total unit cost made up of fixed cost. 9 Three percentages were used 
in the simulation: 15%, 33%, and 50%. The different percentages 
can be viewed as simulating different degrees of operating 
leverage, of capital intensiveness, or of plant automation. The 
15% level reflects the average fraction of price represented by 
fixed overhead in the aerospace industry, as estimated at one time 
by DOD (Balut, 1981). 10 The larger percentages are consistent with 
the trend toward increased automation (McCullough and Balut, 1986) . 

iv) Production Rate Trend (PROTREND) : When initiating a new 
product, it is not uncommon for the production rate per period to 
start low and trend upward to some "normal" level. This may be due 
both to the need to develop demand for the output the desire to 
start with a small production volume, allowing slack for ^eJrking 
bugs out of the production process. Alternatively, when a "new" 
product results from a relatively small modification of an existing 
product, sufficient customer demand or sufficient cor fidence in the 



’Operationally this is a bit complex, since both per unit 
variable and per unit fixed cost depend on other simulation inputs 
(cumulative quantity and production rate per period) . The process 
of relating fixed cost to total cost was as follows: First, a 
cumulative average per unit variable cost for all units produced 
during the estimation period was determined. Then a standard fixed 
cost per unit was set relative to the cumulative average per unit 
variable cost. For example, if standard fixed cost per unit was 
set equal to cumulative average variable cost per unit, then "on 
average" fixed cost would comprise 50% of total unit cost during 
the estimation period. Actual fixed cost per unit may differ from 
standard fixed cost per unit if the production rate (discussed 
later) was not at 100% of standard. 

10 In the absence of firm-specific cost data, the Cost Analysis 
Improvement Group in the Office of the Secretary of Defense treats 
15% of the unit price of a defense system as representing fixed 
cost (Pilling, 1990) . 
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production process may be assumed and full scale production may be 
initiated rapidly. In short, two different patterns in production 
volume may be exhibited early on when introducing a new item: a 
gradual growing trend toward full scale production or a level trend 
due to introduction at full scale production volume. 

How might the production rate trend during the estimation 
period affect the relative prediction accuracy of the LC and the RA 
models? Two contrary arguments seem relevant. On the one hand, 
the purpose of an RA model is to explain variance in cost due to 
variance in production rate. If production rate is virtually level 
during a model estimation period, there would be little period-to- 
period differences in unit cost caused by the spreading of fixed 
costs over varying outputs. Any rate effects would likely be 
swamped by other random impacts on cost. This suggests greater 
incremental benefit to using an RA model when the production rate 
has not been level, i.e., when growth occurred during the model 
estimation period. On the other hand, a growing production rate 
from period to period results in statistical problems. Within the 
RA model, there will be greater colinearity between cumulative 
quantity (necessarily growing each period) and production rate. 
Empirically, this colinearity has been observed by many cost 
analysts (e. g. , Gulledge and Woirer, 1986). The colinearity makes 
production rate a somewhat redundant variable and causes unreliable 
parameter estimates. This suggests less incremental benefit to 
using an RA model if production rate has been growing. 

Which argument holds is an empirical question. The simulation 
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created two production trends during the model estimation period: 
"level" and "growth." These represented general trends (but, as 
will become clear momentarily, variance around the general trend 
was introduced) . The level trend simulated a production rate set 
at a "standard" 100% each period during model estimation. The 
growth trend simulated production rate climbing gradually to 100%. 
Details of the trends are in table 1. 

v) Production Rate Instability/Variance (RATEVAR) : Numerous 
factors, in addition to the general trend in output discussed 
above, may operate to cause period-to-period fluctuations in 
production rate. Manufacturers typically do not have complete 
control over either demand for output or supply of inputs. 
Conditions in either market can cause instability in production 
rate. (Of course, unstable demand, due to the uncertainties of 
annual budget negotiations, is claimed to be a major cause of cost 
growth during the acquisition of major weapon systems by the DoD) . 
The concern here is with the likely effect of rate instability on 
the relative accuracy of LC and RA models. If production rate is 
highly stable across periods, there would be little variance in 
cost outcomes on which to estimate a rate parameter. Rate effects 
would be dominated by other impacts on cost. This suggests greater 
benefit to using an RA model when production rate is unstable over 
time. 

Production rate instability was simulated by adding random 
variance to each period's production rate during the estimation 
period. The amount of variance ranged from a coefficient of 
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variation of .05 to .15 to .25. For example, if the production 
trend was level and the coefficient of variation was .05 then 
"actual" production rates simulated were generated by a normal 
distribution with mean equal to the standard production rate (100%) 
and sigma equal to 5%. 

vi) Cost Noise/Variance (COSTVAR) : From period to period 
there will be unsystematic, unanticipated, non-recurring, random 
factors that will impact unit cost. Changes in the cost, type or 
availability of input resources, temporary increases or decreases 
in efficiency, and unplanned changes in the production process are 
all possible causes. Conceptually such unsystematic factors can be 
thought of as adding random noise to unit cost. While unsystematic 
variation in cost cannot (by definition) be controlled, it is often 
possible to characterize different production processes in terms of 
the degree of unsystematic variation; some processes are simply 
less well-understood, more uncertain, and less stable than others. 

Does the relative predictive accuracy of LC and RA models 
depend on the stability of the process underlying cost? To 
investigate this question, random variance was added to the 
simulated costs generated from the cost function. The amount of 
variance ranged from a coefficient of variation of .05 to .15 to 
.25. For example, when the coefficient of variation was .25, then 
"actual" unit costs simulated were generated by a normal 
distribution with mean equal to cost from equation 9 and sigma 
equal to 25%. 

vii) Future Production Level (FUTUPROD) : Once a model is 
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constructed (from data available during the estimation period), it 
is to be used to predict future cost. The production rate planned 
for the future may vary from past levels. Further growth may be 
planned. Cutbacks may be anticipated. Will the level of the 
future production rate affect the relative predictive accuracy of 
the LC and RA models? Is one model more accurate if cutbacks in 
production are anticipated and another if growth is planned? 
Conflicting scenarios can be argued. For example, on the one hand, 
inclusion of a rate term might be expected to increase prediction 
accuracy when production rate changes significantly (i. e., either 
growth or decline in the future period) . On the other hand, if 
growth during the future prediction period merely continues a 
growth trend established during the estimation period, then 
production rate will once again tend to be correlated with 
cumulative quantity, and the incremental benefit of an RA model 
over an LC model may be less obvious. 

In the simulation, future production was set at three levels: 
low (50% of standard) , same (100% of standard) and high (150% of 
standard) . These simulate conditions of cutting back, maintaining 
or increasing production relative to the level of production 
existing at the end of the model estimation period. 

4. The Cost Objective : What is to be predicted? Up to this 
point the stated purpose of the study has been to evaluate accuracy 
when predicting future cost. But which future cost? Three 
alternatives were examined. 

i) Next period average unit cost: As the label suggests this 
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is the average per unit cost of items in the production "lot" 
manufactured in the first period following the estimation period. 
Here the total cost of producing the output for the period is 
simply divided by the output volume to arrive at unit cost. 
Attention to this cost objective simulates the need to predict near 
term unit cost. 

ii) Total cost over a finite production horizon: The 
objective here is to predict the total cost of all units produced 
during a fixed length production horizon. Three periods were used 
as the length of the production horizon (one production lot 
produced each period) . If the future production rate is low (high) 
then relatively few (many) units will be produced during the finite 
production horizon. Attention to this cost objective simulates the 
need to predict costs over some specific planning period, 
regardless of the volume to be produced during that planning 
period . 

iii) Total program cost: The objective here is to predictive 
total cost for a specified number of units. If the future 
production rate is low (high) then relatively more (fewer) periods 
will be required to manufacture the desired output. The simulation 
was constructed such that at a low (same, high) level of future 
production six (three, two) future periods were required to produce 
the output. Attention to this cost objective simulates the need to 
predict total cost for a particular production program, regardless 
of the number of future periods necessary to complete the program. 

Examining each of these three cost objectives was deemed 
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necessary to provide a well-rounded investigation of predictive 
accuracy. However, the findings were substantially the same across 
the three cost objectives. In the interests of space, the 
remainder of this paper will discuss the analysis and results only 
for the first cost objective, the average cost per unit for the 
next period's output. 

5. The Measures of Prediction Performance : In order to 
compare prediction accuracy between LC and RA models, two kinds of 
measures were required. First, a model prediction error (ERR) was 
determined separately for each (LC or RA) model as follows: 

ERR = |PUC - AUC | + AUC 



where \ 

PUC = Predicted unit cost from either the learning curve 
or the rate adjustment model. 

AUC = Actual unit cost as generated by the cost 
function. 

This is a commonly used error measure, the absolute percentage 
error. 

Next a simple difference in errors (ERRDIFF) between the LC 
prediction and the RA prediction was calculated. 



ERRDIFF = ERR lc - ERR ra 

where 



ERR lc = Absolute percentage error using the learning curve 
model prediction of cost. 

ERR ra = Absolute percentage error using the rate 
adjustment model prediction of cost. 

Positive values for ERRDIFF mean that the rate adjustment model 

produced smaller prediction errors that the learning curve model 
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and consequently imply increased accuracy from incorporating a 
production rate variable into the prediction process. Negative 
values mean the learning curve model is more accurate. ERRDIFF 
represents the dependent variable in the statistical analysis. The 
research question then becomes: What factors or conditions explain 
variance in ERRDIFF? 

Figure 1 summarizes the complete simulation process leading up 
to the determination of ERRDIFF. The simulation was run once for 
each possible combination of treatments. Given seven factors 
varied and three possible values for each factor (except for 
PROTREND which had two) , there were 3x3x3x3x3x3x2= 
1458 combinations. Thus the simulation generated 1458 observations 
and 1458 values for ERRDIFF. 11 



ANALYSIS AND FINDINGS 

The simulation results were evaluated using analysis of 
variance (ANOVA) to conduct tests of statistical significance. All 
main effects are discussed in the following section. First order 
(pairwise) interactions were also tested. Those significant at .01 



n In the simulation, just as in the real practice of cost 
analysis, it is possible for a model estimated on limited data to 
be very inaccurate, leading to extreme values for ERR (or ERRDIFF) . 
If such outlier values were to be used in the subsequent analysis, 
findings would be driven by the outliers. Screening of the 
observations for outliers was necessary. During the simulation, if 
a model produced an ERR value in excess of 100%, then that value 
was replaced with 100%. This truncation has the effect of reducing 
the impact of an outlier on the analysis while still retaining the 
observation as one that exhibited poor accuracy. Alternative 
approaches to the outlier problem included deletion instead of 
truncation and use of a 50% ERR cutoff rather than the 100% cutoff. 
Findings were not sensitive to these alternatives. 
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FIGURE 1 

SIMULATION FLOWCHART 




are also discussed. Table 2 provides the ANOVA results. 



Main Effects : As indicated in table 2, all main effects, 
except LEARNRAT, are significant at .0005, indicating that values 
for ERRDIFF are significantly influenced by the treatment 
conditions. Table 3 summarizes ERRDIFF values under the various 
experimental conditions. 

The major findings are perhaps made most obvious by a plot of 
ERRDIFF by treatments. Figure 2 shows a sample plot of ERRDIFF 
against DATAHIST. Two points are of interest. First, the relative 
accuracy of the LC and RA models clearly does depend on the number 
of observations available for model estimation. Second, both 
negative and positive value for ERRDIFF are present. This implies 
that under some conditions (few observations) the LC model is more 
accurate than the RA model. But under different conditions (larger 
number of observations) the RA model is more accurate. The 
crossover point occurs at about seven observations. 

Figure 3 shows a similar plot of ERRDIFF with all (significant) 
variables superimposed. In this plot, 1, 2, and 3 on the x-axis 
reflect low, medium, and high values for the independent variables 
(which are taken from the left, middle and right columns of Table 
3). Figure 3 makes several points. First for each the six 
variables plotted, ERRDIFF varies significantly depending on the 
value (treatment) for the independent variable. This simply 
implies that the experimental conditions do reflect factors 
relevant to model accuracy. 

Second for all six independent variables, at some point over 
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TABLE 2 



ANALYSIS OF VARIANCE RESULTS 



SOURCE 


DF 


SUM OF SOUARES 


MEAN SOU ARE 


F VALUE 


Model 


85 


33.10921 


.3895 


9.03 


Error 


1372 


59.18006 


.0431 


PR>F : 


Corrected Total 


1457 


92.28927 




. 0001 



R 2 


CV 


ERRDIFF MEAN 


.3588 


1236.81 


-.0168 



SOURCE 


DF 


ANOVA SS 


F VALUE 


PR>F 


DATAHIST 


2 


3.6780 


42.63 


. 0001 


LEARNRAT 


2 


.0510 


. 59 


.5536 


BURDEN 


2 


.7383 


8.56 


. 0002 


PROTREND 


1 


1.6259 


37.70 


. 0001 


RATEVAR 


2 


5.4728 


63.44 


. 0001 


COSTVAR 


2 


9.3918 


108.87 


. 0001 


FUTUPROD 


2 


.6662 


7.72 


. 0005 


DAT AH 1ST* LEARNRAT 


4 


.0699 


.41 


.8050 


DATAHIST* BURDEN 


4 


. 1772 


1.03 


. 3920 


DAT AH 1ST* PROTREND 


2 


. 3217 


3.73 


. 0243 


DAT AH I S T * RATE VAR 


4 


.0844 


.49 


.7436 


DATAH I ST* COSTVAR 


4 


.9813 


5.69 


. 0002 


DATAH I ST* FUTUPROD 


4 


.7531 


4.37 


.0016 


LEARNRAT* BURDEN 


4 


.2444 


1.42 


. 2260 


LEARNRAT* PROTREND 


2 


.0092 


. 11 


. 8979 


LEARNRAT* RATEVAR 


4 


. 3933 


2.28 


. 0588 


LEARNRAT * COS TVAR 


4 


.2407 


1.40 


.2332 


LEARNRAT* FUTUPROD 


4 


.3818 


2.21 


. 0655 


BURDEN* PROTREND 


2 


. 0806 


.93 


. 3931 


BURDEN* RATEVAR 


4 


.2325 


1.35 


.2501 


BURDEN* COSTVAR 


4 


.7304 


4.23 


.0021 


BURDEN* FUTUPROD 


4 


1.0539 


6.11 


. 0001 


PROTREN D * RATEVAR 


2 


2.1867 


25.35 


.0001 


PROTREND* COSTVAR 


2 


. 1022 


1.19 


. 3059 


PROTREND* FUTUPROD 


2 


.4751 


5.51 


. 0041 


RATEVAR* COSTVAR 


4 


.3163 


1.83 


. 1199 


RATEVAR* FUTUPROD 


4 


1.7734 


10.28 


. 0001 


COSTVAR* FUTUPROD 


4 


.8760 


5.08 


. 0005 



TABLE 3 



PREDICTION ERROR DIFFERENCE BETWEEN 
LEARNING CURVE AND RATE ADJUSTMENT MODELS: 

MAIN EFFECTS 



Independent Variable 


ERRDIFF for Each 


Level 


DATAHIST Value: 


4 


7 


10 


ERRDIFF Mean: 


-.87 


.007 


. 030 


LEARNRAT Value: 


75% 


85% 


95% 


ERRDIFF Mean: 


-.009 


-.019 


-.023 


BURDEN Value: 


15% 


33% 


50% 


ERRDIFF Mean: 


-.047 


-.009 


.006 


PROTREND Value: 


level 


- 


growth 


ERRDIFF Mean: 


-.050 


\ 


.017 


RATEVAR Value: 


. 05 


. 15 


.25 


ERRDIFF Mean: 


-.095 


-.010 


. 054 


COSTVAR Value: 


IT) 

o 

• 


. 15 


.25 


ERRDIFF Mean: 


. 082 


-.019 


-. 114 


FUTUPROD Value 


low 


same 


high 


ERRDIFF Mean: 


.003 


-.007 


-.046 



FIGURE 2 




DATA HISTORY 



FIGURE 3 



PLOT OF PREDICTION ERROR DIFFERENCE 
BY MAIN EFFECTS 
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FUTUPROO - 7 



EXPERIMENTAL CONDITIONS 



the range of these variables, both positive and negative values for 
ERRDIFF exist. This means that neither the LC model nor the RA 
model dominates. Each model is more accurate under some 
experimental conditions. 

Third, some factors apparently have a greater impact on 
relative prediction accuracy than others. For example, the plot 
shows that both the most extreme values for ERRDIFF (highest 
positive and lowest negative) are associated with different 
treatments on COSTVAR. The F-value for COSTVAR is also the highest 
and most significant in the ANOVA. This suggests that the greatest 
impact on relative predictive accuracy is due to variation in the 
amount of unsystematic noise impacting the measure of cost. A 
relatively large effect is also apparent due to variation in 
RATEVAR; while relatively small (but still significant) effects are 
associated with variations in BURDEN and FUTUPROD. These 
observations are interesting but must be interpreted with caution. 
COSTVAR does have the greatest impact on ERRDIFF over the 
particular range of treatments examined by the simulation. But if 
the ranges over which the individual independent variables were 
allowed to vary were altered, a different factor could appear to be 
most important. 

' Fourth, values for ERRDIFF are either monotonically increasing 
or decreasing for the various independent variable treatments. 
Hence some general conclusions about the impact of the experimental 
conditions on relative prediction errors are possible: 

a) Data History: As indicated above the RA model outperforms 
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the LC model when the number of observations for model estimation 



is relatively greater. This is expected; the benefit from greater 
precision due to the additional term in the RA model can only be 
realized when the data are sufficient to produce reliable parameter 
estimates . 

b) Fixed Cost Burden: As expected the RA model tends to 
outperform the LC model as the proportion of total unit cost 
comprised of fixed cost increases. This implies that the benefit 
derived from use of a rate adjustment model depends on the capital 
intensiveness of the production process. Surprising, perhaps, was 
the fact that the change in ERRDIFF from the low (15%) fixed cost 
treatment to the high (50%) was relatively small. 

c) Past Production Trend: The RA model tends to outperform 
the LC model if the production rate was growing during the model 
estimation period. This finding suggests that it is necessary to 
have a changing production rate from period-to-period in order for 
the model estimation process to reliably capture the effect of rate 
on cost. It also suggests that relatively higher correlation 
between cumulative quantity and production rate (which will occur 
when rate has been growing) does not imply that cumulative quantity 
will be a sufficient variable. Cumulative quantity does not 
adequately capture the impact of rate changes. 

d) Production Rate Instability: The RA model tends to 
outperform the LC model when there is greater instability in 
production rate. This suggests that it is desirable to have large 
period-to-period variation in production rate during the time of 
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model estimation. In retrospect this conclusion seems obvious. If 
the variance in cost caused by rate changes is small, because the 
rate changes themselves are small, then the effect will be swamped 
by other effects and model parameters will be unreliable. 

e) Cost Noise/Variance: As the random noise in unit cost 
increases, the RA model will perform more poorly relative to the LC 
model. This suggests that when there are unsystematic, 
unpredictable factors that influence cost, use of the RA model is 
less beneficial. The noise leads to unreliable estimates of the 
production rate effect and poorer cost predictions. 

f) Future Production Level: The degree to which the RA model 
outperforms the LC model appears to be inversely associated with 
the production rate in the prediction period. When cutbacks in 
production are anticipated, cost is more accurately predicted by 
the RA model. When growth is anticipated, cost is more accurately 
predicted by the LC model. This suggests that the RA model is most 
beneficial when a reversing of the production trend from the 
estimation period to the prediction period is expected. 

Interaction Effects : Eight first order interactions were also 
found to be significant in the ANOVA. Note that five of these 
interactions involve FUTUPROD as one of the interacting variables. 
This means that the impact of other variables on relative 
prediction accuracy is particularly sensitive to the anticipated 
level of production in the prediction period. Examining these 
eight interactions individually leads to further insight concerning 
the conditions that impact relative prediction accuracy. They are 
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illustrated in Figures 4 through 11. 

Figure 4, the interaction of Data History with Future 
Production Level, demonstrates again that the RA model tends to 
outperform the LC model as the data history becomes richer. But 
this effect is most pronounced when a low future production level 
is planned. If a cutback in production is projected and few 
observations are available for model estimation, the LC model is 
substantially more accurate than the RA model (the most negative 
value for ERRDIFF occurs under these conditions) . If a cutback in 
production is projected and relatively many observations are 
available for model estimation, the RA model is substantially more 
accurate than the LC model. (The most positive vais.e for ERRDIFF 
occurs under these conditions.) 

Figure 5, the interaction of Fixed Cost Burden and Future 
Production Level, demonstrates a somewhat analogous effect. The RA 
model tends toward better accuracy than the LC model as fixed cost 
burden increases. But again this effect is most pronounced when 
low levels of production in the future are planned. The analyst's 
concern for the impact of burden percentage on model accuracy 
should be greatest when volume cutbacks are expected. 

Figure 6, the interaction of Production Rate Instability and 
Future Production Level also shows the importance of the future 
production level. The RA model tends to outperform the LC model as 
the degree of variability in the production rate increases. But 
the impact of rate variability is most pronounced when a cutback in 
production volume is planned. If the production rate was stable 
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FIGURE 4 
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during the model estimation period, the LC model is more accurate. 
If the production rate was fluctuating and a cutback in volume is 
planned, then the RA model substantially outperforms the LC model. 

Figure 7, the interaction of Cost Noise with Future Production 
Level, again illustrates how the future production volume impacts 
relative prediction accuracy. For all treatments, as unsystematic 
factors make cost more noisy, the RA model tends to lose its 
relative advantage over the simpler LC model. But this 
deterioration in predictive accuracy due to noise for the RA model 
is more evident when a cutback in future production volume is 
anticipated. 

Figure 8 shows the interaction of the Production Trend during 
the estimation period and the Future Production Level during the 
prediction period. Again the steepest curve occurs when the future 
production level is low. This plot demonstrates that if production 
rate is relatively level during the model estimation period, the 
simple LC model tends to outperform the RA model. But the RA model 
outperforms the LC model when there is a shift or change in 
direction of the production trend from the estimation period to the 
prediction period. This follows from the positive value for 
ERRDIFF when a growing past production trend is coupled with a 
decline in future production level. 

Figure 9 shows an interesting interaction between Cost Noise 
and Fixed Cost Burden. The plot shows that under all conditions, 
increasing cost noise reduces, and then eliminates, any benefit 
from using the RA model. But the deterioration is most dramatic if 
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the percentage of fixed cost in total cost is high. If burden is 
high and cost is not subject to unsystematic noise, then the RA 
model is substantially superior. But if burden is high and cost 
is subject to unsystematic noise, the RA model is substantially 
inferior. 

Figure 10, the interaction between Cost Noise and Data History, 
illustrates a somewhat additive effect of two factors. Either 
increasing cost noise or reducing the number of observations causes 
deterioration in the performance of the RA model relative to the LC 
model. But combining the two effects magnifies the deterioration. 
If there is high cost noise and few observations on which to 
estimate model parameters, the RA model becomes ver} « unreliable and 
the LC model is strongly superior. 

Finally, Figure 11 shows the interaction between Production 
Rate Instability and Production Rate Trend. These variables 
reflect two aspects of production rate during the model estimation 
period. The plot illustrates that when there is little trend in 
rate, coupled with little variability in rate relative to trend, 
then there is little basis on which to estimate a rate parameter. 
Hence the LC model strongly outperforms the RA model. 

SUMMARY AND CONCLUSIONS 

The central research question addressed in the study was: 
under what conditions does consideration of production rate and 
incorporation of a rate variable into a learning curve analysis of 
cost lead to a more accurate prediction of future cost? The 
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analysis simulated prediction for both a traditional learning curve 
and a rate adjustment model and compared prediction accuracy under 
various conditions. One central finding was that neither a 
traditional learning curve analysis nor an analysis which considers 
the impact of production rate on cost is inherently superior. 
Neither models dominates; both outperform the other under some 
conditions. 

General tendencies were evident. Considering production rate 
in the analysis lead to reduction in prediction error and improved 
accuracy when 

The number of observations available for the analysis 
was relatively rich. 

The amount of fixed cost in total cost was relatively 
high. 

The production rate trend had been growing during the 
model estimation period. 

The period-to-period variability in production rate was 
relatively large. 

— Random noise in cost due to unsystematic factors 
impacting cost was relatively low. 

Production volume was expected to be cutback in the 
future periods for which cost predictions were being 
made. 

Each of these findings suggests that researchers or cost analysts, 
engaged in a cost prediction or cost analysis problem, may benefit 
from attending to such factors when deciding on the form of model 
or analysis they might bring to bear. 

Numerous interacting impacts of combinations of factors on 
prediction accuracy were also evident, but a broad "theme" was 
apparent in the interactions, suggesting a general conclusion: 
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The greatest impact (of changes in the various factors) 
on relative prediction accuracy (of the learning curve 
approach and the rate adjustment approach) occurs when 
cutbacks in future production are anticipated. 

This means that researchers and cost analysts, attempting to 

predict future cost in an environment where future production 

volume is declining, will find the choice of an approach to be most 

critical. The relative accuracy of the learning curve approach or 

the rate adjustment approach is particularly sensitive to changes 

in data richness, fixed cost burden, production rate trend and 

stability, and cost noise when cutbacks are anticipated. 

Other interactions collectively suggest a second general 
conclusion: 

I 

The impacts of factors on the relative prediction 
accuracy of the two approaches tend to be additive. 

This means that if the presence of one factor (say, lean data 

history) and the presence of another (say, high cost noise) both 

reduce the relative accuracy of the rate adjustment approach, then 

the presence of both will magnify the effect. 

The conclusions of any study must be tempered by any 

limitations. The most prominent limitation of this study is the 

use of simulated data. Use of the simulation methodology was 

justified by the need to create a wide range of treatments and 

maintain control over extraneous influences. This limitation 

suggests some directions for future research. 

Re-analyze the research question while altering aspects 
of the simulation methodology. For example, are 
findings sensitive to the cost function assumed? 

Address the same research question using actual cost 
and production rate data. Are the same findings 
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evident when using "real-world" data? 

Providing confirmation of the findings by tests using alternative 

approaches would be beneficial. 

Additional future research may be directed toward new, but 

related, research questions. 

Investigate other qualities of predictive performance, 
for example, bias. Do learning curves or rate 
adjustment models systematically provide predictions 
that are biased toward under or over estimation of 
future cost? Under what conditions? 

Investigate the magnitude of prediction errors using 
either a learning curve or rate adjustment approach. 
How large are average prediction errors under varying 
circumstances? 

Investigate other competing models or approaches to 
cost prediction. Perhaps accuracy can be improved by 
using some version of a "moving average" prediction 
model. Can such a model outperform both the learning 
curve and the rate adjustment approach? If so, under 
what circumstances? 
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